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- ■ Abstract 

o 

Cn ' In this paper, we study the integraUty gap of the Knapsack hnear program in the Sherah- 

Adams and Lasserre hierarchies. First, we show that an integrahty gap of 2 — e persists up 
to a hnear number of rounds of Sherah- Adams, despite the fact that Knapsack admits a fully 
polynomial time approximation scheme |271I33| . Second, we show that the Lasserre hierarchy 
closes the gap quickly. Specifically, after t rounds of Lasserre, the integrality gap decreases to 
t/{t — 1). To the best of our knowledge, this is the first positive result that uses more than a 

\^ ' small number of rounds in the Lasserre hierarchy. Our proof uses a decomposition theorem for 

^^ . the Lasserre hierarchy, which may be of independent interest. 

O . 

1 Introduction 

^ ■ Many approximation algorithms work in two phases: first, solve a linear programming (LP) or 

f^ . semi-definite programming (SDP) relaxation; then, round the fractional solution to obtain a feasible 

pg ', integer solution to the original problem. This paradigm is amazingly powerful; in particular, under 

the unique game conjecture, it yields the best possible ratio for MaxCut and a wide variety of other 
[~^ . problems, see e.g. [37] . 

^P I However, these algorithms have a limitation. Since they are usually analyzed by comparing 

the value of the output to that of the fractional solution, we cannot generally hope to get a 
better approximation ratio than the integrality gap of the relaxation. Furthermore, for any given 
combinatorial optimization problem, there are many possible LP/SDP relaxations, and it is difficult 
K> ' to determine which relaxations have the best integrality gaps. 

j^ ■ This has lead to efforts to provide systematic procedures for constructing a sequence of increas- 

ingly tight mathematical programming relaxations for 0-1 optimization problems. A number of 
different procedures of this type have been proposed: by Lovasz and Schrijver [3l], Sherali and 
Adams [H], Balas, Ceria and Cornuejols [6j, Lasserre |30p31j and others. While they differ in the 
details, they all operate in a series of rounds starting from an LP or SDP relaxation, eventually 
ending with an exact integer formulation. The strengthened relaxation after t rounds can typically 
be solved in n^^'-) time and, roughly, satisfies the property that the values of any t variables in the 
original relaxation can be expressed as the projection of a convex combination of integer solutions. 
A major line of research in this area has focused on understanding the strengths and limitations 
of these procedures. Of particular interest to our community is the question of how the integrality 
gaps for interesting combinatorial optimization problems evolve through a series of rounds of one of 
these procedures. On the one hand, if the integrality gaps of successive relaxations drop sufficiently 
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fast, there is the potential for an improved approximation algorithm (see [8|lll t [T7 1 ll8) for example). 
On the other hand, a large integrality gap persisting for a large, say logarithmic, number of rounds 
rules out (unconditionally) a very wide class of efficient approximation algorithms, namely those 
whose output is analyzed by comparing it to the value of a class of LP/SDP relaxations. This im- 
plicitly contains most known sophisticated approximation algorithms for many problems including 
SparsestCut and MaximumSatisifiability. Indeed, serveral very strong negative results of this type 
have been obtained (see [IlElEllIllOlalllSSllMlEHHinillS] and others). These are also viewed as 
lower bounds of approximability in certain restricted models of computation. 

How strong are these restricted models of computation? In other words, how much do lower 
bounds in these models tell us about the intrinsic hardness of the problems studied? To explore 
this question, we focus on one problem that is well-known to be "easy" from the viewpoint of 
approximability: Knapsack. We obtain the following results: 

• We show that an integrality gap close to 2 persists up to a linear number of rounds of Sherali- 
Adams. (The integrality gap of the natural LP is 2.) 

This is interesting since Knapsack has a fully polynomial time approximation scheme |27il33j . 
This confirms and amplifies what has already been observed in other contexts (e.g [14j): the 
Sherali-Adams restricted model of computation has serious weaknesses: a lower bound in this 
model does not necessarily imply that it is difficult to get a good approximation algorithm. 

• We show that Lasserre's hierarchy closes the gap quickly. Specifically, after t rounds of 
Laserre, the integrality gap decreases to t/{t — 1). 

It is known that a few rounds of Lasserre can yield better relaxations. For example, two 
rounds of Lasserre applied to the MaxCut LP yields an SDP that is at least as strong as that 
used by Goemans and Williamson to get the best known approximation algorithm, and the 
SDP in [3] which leads to the best known approximation algorithm for SparsestCut can be 
obtained by three rounds of Lasserre. However, to the best of our knowledge, this is the first 
positive result that utilizes more than a small constant number of rounds in the Lasserre 
hierarchy. 

1.1 Related Work 

Many known approximation algorithms can be recognized in hindsight as starting from a natural 
relaxation and strengthening it using a couple of levels of lift-and-project. The original hope [2\ 
had been to use lift and project systems as a systematic approach to designing novel algorithms 
with better approximation ratios. Instead, the last few years have mostly seen the emergence of a 
multitude of lower bounds. Indeed, lift and project systems have been studied mostly for well known 
difficult problems: MaxCut [HKISliO], SparsestCut, [TT.TS] VertexCover [Tti3l[T3|[2n[22l[26t[4nll42]. 
HypergraphVertexCover, TSP [1^, MaximumAcyclicSubgraph ^14j, CSP [391133]. and more. 

The Knapsack problem |28,35] has a fully polynomial time approximation scheme |27y 33j. The 
natural LP relaxation (to be stated in full detail in the next section) has an integrality gap of 
2 — e [28j. Although we are not aware of previous work on using the lift and project systems for 
Knapsack, the problem of strengthening the LP relaxation via addition of well-chosen inequalities 
has been much the object of much interest in the past in the mathematical programming community, 
as stronger LP relaxations are extremely useful to speed up branch-and-bound heuristics. The 
knapsack polytope was studied in detail by Weismantel [35]. Valid inequalities were studied in 
[5|[7l l23l[24ll45j . In particular, whenever 5 is a minimal set (w.r.to inclusion) that does not fit 
in the knapsack, then "^suij-Wii^Swyw-y^J — I*?! — 1 is a valid inequality. Generalizations and 



variations were also studied in |20 y 25 l B6]. Thus, in spite of the existence of a dynamic program to 
solve the problem, Knapsack is fundamental enough that understanding the polytope (and its lifted 
tightenings) is of intrinsic interest. In [lOj, Bienstock formulated LP with arbitrary small integrality 
gaps for Knapsack using "structural disjunctions", and asked if the popular hierarchies reduce the 
gap of the Knapsack linear program. Our results give a negative answer for Sherali- Adams and a 
strong affirmative one for Lasserre. 

Our results confirm the indication from |291l38j for example that the Sherali-Adams lift and 
project is not powerful enough to be an indicator of the hardness of problems. However, it should 
be noted that if the problem was phrased as a decision problem and the objective function was 
replaced by an additional constraint of the constraint polytope, then Sherali-Adams would succeed 
in reducing the integrality gap; thus the choice of the initial LP formulation is critical. On the 
other hand, little is know about the Lasserre hierarchy, as the first negative results were about 
/c-CSP [39p43j. Our positive result leaves open the possibility that the Lasserre hierarchy may have 
promise as a tool to capture the intrinsic difficulty of problems. 

2 Preliminaries 

2.1 The Knapsack problem 

Our focus in this paper is on the Knapsack problem. In the Knapsack problem, we are given a set 
of n objects V = [n] with sizes ci, C2, . . . c„, values vi,V2, ■ ■ -Vn, and a capacity C. We assume that 
for every i, Ci < C. The objective is to select a subset of objects of maximum total value such that 
the total size of the objects selected does not exceed C. 

The standard linear programming (LP) relaxation |28j for Knapsack is given by: 

max > ViXi 
iev 

Yj Ci^^i < C (1) 

S.t. < iGV 

0<Xi<l \/i£V 

The intended intepretation of an integral solution of this LP is obvious: Xj = 1 means the object i 
is selected, and Xj = means it is not. The constraint can be written as g{x) = C — Y2- CiXi > 0. 

Let Greedy denote the algorithm that puts objects in the knapsack by order of decreasing ratio 
Vi/ci, stopping as soon as the next object would exceed the capacity. The following lemma is 
folklore. 

Lemma 1 Consider an instance {C,V) of Knapsack and its LP relaxation K given by ^^. Then 

Value{K) < Value (Greedy{C,V)) +maxt'j. 

2.2 The Sherali-Adams and Lasserre hierarchies 

We next review the lift-and-project hierarchies that we will use in this paper. The descriptions we 
give here assume that the base program is linear and mostly use the notation given in the survey 
paper by Laurent [32]. To see that these hierarchies apply at a much greater level of generality we 
refer the reader to Laurent's paper [32] . 



Let K he a polytope defined by a set of linear constraints gi,g2, ■ ■ ■ dm'- 

K = {x £ [0, l]"|c/^(x) > Ofor £= 1,2, ...m}. (2) 

We are interested in optimizing a linear objective function / over the convex hull P = conv (i^n{0, l}") 
of integral points in K. Here, P is the set of convex combinations of all integral solutions of the 
given combinatorial problem and K is the set of solutions to its linear relaxation. For example, if 
K is defined by ([1]), then P is the set of convex combinations of valid integer solutions to Knapsack. 

If all vertices of K are integral then P = K and we are done. Otherwise, we would like to 
strengthen the relaxation K by adding additional valid constraints. The Sherali-Adams (SA) and 
Lasserre hierarchies are two different systematic ways to construct these additional constraints. In 
the SA hierarchy, all the constraints added are linear, whereas Lasserre's hierarchy is stronger and 
introduces a set of positive semi-definite constraints. However, for consistency, we will describe 
both hierarchies as requiring certain submatrices to be positive semi-definite (readers who are not 
familiar with the following formulation of SA are referred to Appendix [B] for a linear formulation 
of the hierarchy.) 

To this end, we first state some notation. Throughout this paper we will use V {V) to denote 
the power set of V ^ and Vt iV) to denote the collection of all subsets of V whose sizes are at most 
t. Also, given two sets of coordinates T and S, T Q S and y G R'^ , by ii\t we denote the projection 
of y onto T. 

Next, we review the definition of the shift operator between two vectors x,y £ R^^^': x *y is a. 
vector in B/^' such that 

(x *y)i ='^ xjyiuj. 
jcy 

Lemma 2 ( |32j ) The shift operator is commutative: for any vectors x,y,z € R^^'^', we have 
X * {y * z) = y * (x * z). 

A polynomial P{x) = Ylicv '^i Wi&i ^i ^^^ ^^^° ^^ viewed as a vector indexed by subsets of V. 
We define the vector P * y accordingly: (P * y)j = J2jcv '^JViuJ- 

Finally, let T be a collection of subsets of V and y be a vector in R^. We denote by M'j-{y) 
the matrix whose rows and colums are indexed by elements of T such that 

{MTiy))jj = yivjj. 

The main observation is that if x G -RTljO, 1}" then (y/) = (nig/^i) satisfies M'p(^y^{y) = 
yy'^>z and M'p(y^{gi*y) = gi{x)yy'^>z for all constraints gi. Thus requiring principal submatrices 
of these two matrices to be positive semi-definite yields a relaxation. 

Definition 3 For any I < t < n, the t-th Sherali-Adams lifted polytope SA* (K) is the set of 

vectors y G [0, 1]^*(^) such that 2/0 = 1, M-p(^ij^{y)>z and M-p(yi/-^{gi * y)>z for all £ and subsets 
U,W ^V such that \U\ <t and \W\ <t - 1. 

We say that a point x G [0, 1]"" belongs to the t-th Sherali-Adams polytope sa* (K) iff there 
exists a y £ SA* (K) such that yu\ = Xi for all i G 



n\ 



Definition 4 For any I < t < n, the t-th Lasserre lifted polytope La* (K) is the set of vectors 
y G [0, l]^2t(V) such that y^ = I, Mp^(y)(y)^ and Mp^_^(y)(g^ * y)^ for all L 

We say that a point x G [0, 1]" belongs to the t-th Lasserre polytope la* {K) if there exists a 
y G La* {K) such that y^j} = Xi for all i £ V . 



Note that M-pfij\{y) has at most 2 rows and columns, which is constant for t constant, whereas 
M-p^(v){y) has ("i) rows and columns. 

It is immediate from the definitions that sa*+^ (K) C sa* (K), and la*"*"^ (K) C la* (K) for ah 
1 < t < n — 1. Sherali and Adams [41j show that sa" (K) = P, and Lasserre |301I31) show that 
la" (K) = P. Thus, the sequences 

K D sa^ (K) 5 sa^ (K) D • • • D sa" (K) = P 
K D la^ {K) 5 la^ (i^) D • • • D la" {K) = P 

define hierarchies of polytopes that converge to P. Furthermore, the Lasserre hierarchy is stronger 
than the Sherali- Adams hierarchy: la" [K) C sa" {K). In this paper, we show that for the Knapsack 
problem, the Lasserre hierarchy is strictly stronger. 

3 Lower bound for the Sherali- Adams hierarchy for Knapsack 

In this section, we show that the integrality gap of the i-th level of the Sherali- Adams hierarchy for 
Knapsack is close to 2. This lower bound even holds for the uniform Knapsack problem, in which 
Vi = Ci = 1 for all i Q|. 

Theorem 5 For every e,6 > 0, the integrality gap at the t-th level of the Sherali- Adams hierarchy 
for Knapsack where t < Sn is at least (2 — e)(l/(l + 6)). 

Proof. (Sketch - for full proof see Appendix lAl) Consider the instance K of Knapsack with n 
objects where Ci = Vi = 1 for all i € ^ and capacity C = 2(1 — e). Then the optimal integer value 
is 1. On the other hand, we claim that the vector y where y0 = 1, y^ij = C/{n + (t — 1)(1 — e)) and 
yi = for all [/[ > 1 is in SA (K). Thus, the integrality gap of the tth round of Sherali- Adams is 
at least Cn/{n -|- (i - 1)(1 - e)), which is at least (2 - e)(l/(l + 5)) when t < 6n. u 

4 A decomposition theorem for the Lasserre hierarchy 

In this section, we develop the machinery we will need for our Lasserre upper bounds. It turns out 
that it is more convenient to work with families {z ) of characteristic vectors rather than directly 
with y. We begin with some definitions and basic properties. 

Definition 6 (extension) Let T be a collection of subsets of V and let y be a vector indexed by 
sets ofT. We define the extension of y to be the vector y' , indexed by all subsets of V , such that 
y'j equals yi if I £ T and equals otherwise. 

Definition 7 (characteristic polynomial) Let S be a subset of V and X a subset of S. We 
define the characteristic polynomial P of X with respect to S as 

p''ix)=iix, i[{i-x,)= y: (-i)"^^'n-'- 

iex jes\x j-.xcjcs ieJ 



^Some people call this problem Unweighted Knapsack or Subset Sum. 



Lemma 8 (inversion formula) Let y' be a vector indexed by all subsets ofV. Let S be a subset 
of V and, for each X subset of S, let z = P * y' : 



Then y' = Y^xcs ^" 



J:XCJCS 



Proof. Fix a subset / of V. Substituting the definition of zf in X^xc5 ■^i ^ ^^"^ changing the index 
of summation, we get 

XCS ACS J C A 

For ^ 7^ the inner sum is 0, so only the term for A = ^, which equals y'j, remains. ■ 

Lemma 9 Let y' be a vector indexed by all subsets of V , S be a subset of V and X be a subset of 

S. Then 

Zj = Zj^-^ for all I 

zf = z^ iflCX 



Proof. Let /' = / \ X and /" = LnX. Using the definition of zf and noticing that X U L" = X 
yields zf = zf . This immediately implies that for / C X, zf = z^ . 

Finally, consider a set / that intersects S \ X and let i G Lr\{S\X). In the definition of zf , 
we group the terms of the sum into pairs consisting of J such that i ^ J and of J U {i}. Since 
/ = / U {i}, we obtain: 



J-.XCJCS J:XCJCS\{i} 



Corollary 10 Let y' be a vector indexed by all subsets of V , S be a subset of V and X be a subset 
of S. Let w^ be defined as z^ / zf if zf 7^ and defined as otherwise. Then, if zf 7^ 0, then 
wfx equals 1 for elements of X and for elements of S\X. 



Definition 11 (closed under shifting) Let S be an arbitrary subset ofV and T be a collection 
of subsets of V . We say that T is closed under shifting by S if 

Y £T ^ yx CS, XUY eT. 

The following lemma generalizes Lemma 5 in [32j. It proves that the positive-semidefinite property 
carries over from y to (z^). 

Lemma 12 Let S be an arbitrary subset of V and T be a collection of subsets of V that is closed 
under shifting by S. Let y be a vector indexed by sets ofT. Then 

Mr{y)h ^ yx cs, Mr{z^)h 0. 



Proof. Since M-j-{y)>z 0, there exist vectors vj, I &T, such that {vi,vj) = yiuj- Fix a subset X 
of S. For each I & T, let 

wi = X] (-l)'^''^/uxuH, 

which is weh-defined since T is closed under shifting by S. 

Let I,J^T. It is easy to check that {wi,wj) = {z^)i\jj. Indeed, 

{wi,wj)= Y, Y. (-1)'^'^'^'(^/uxu//,^^juxul) (3) 

HCS\X LCS\X 

= E E (-l)""+"''y/uJuxuHuL (4) 

Hcs\xhcs\x 

by definition of vj , f j and since T is closed under shifting by S (so that this is well-defined) . 
Consider a non-empty subset H of S\X and let z G /7. We group the terms of the inner sum into 
pairs consisting of L such that i ^ L and of L U {i}. Since H = H L) {i}, we obtain: 

E (-l)l^l + l^ly/UJUXUHUL = Yl ((-l)l^l + l'^l + (-1)1^1 + 1^1+^) yiuJUXUHUL = 0. 

LCS\X LC{S\X\{i}) 

Thus, the expression in (j4]) becomes 

{wi,Wj) = ^ (-l)''^'y/UJUXUL = (2'^)/UJ- 
LCS\X 

This implies that Mr{z^)^ 0. ■ 

In the rest of the section, we prove a decomposition theorem for the Lasserre hierarchy, which 
allows us to "divide" the action of the hierarchy and think of it as using the first few rounds on 
some subset of variables, and the other rounds on the rest. We will use this theorem to prove that 
the Lasserre hierarchy closes the gap for the Knapsack problem in the next section. 

Theorem 13 Let t > 1 and y € La {K). Let k < t and S be a subset of V and such that 

\lnS\ >k =^ yi = 0. (5) 

Consider the projection y\v2t~2k(V) ^f V ^'^ ^^^ coordinates corresponding to subsets of size at most 
2t — 2k of V. Then there exist subsets Xi,X2, ■ ■ ■ ,Xm of S such that y\'P2t-2k(V) ^^ '^ convex 
combination of vectors vo "■ with the following properties: 

"^m 1o tfjeS\Xf, 



w 



^' e La*-'^ {K); and 



^i f„^ ^ r- Q +h^^ n„Xi\_ ,.., _ r- I -.i-k 



• if Ki is obtained from K by setting Xj = w^\ for j G S, then w^ |-P2t-2fc(V\S') ^ ^a (Li^i)- 

To prove Theorem 1 13^ we will need a couple more lemmas. In the first one, using assumption ([5]), 
we extend the positive semi-definite properties from y to y', and then, using Lemma [T^ from y' to 



Lemma 14 Let t,y,S,k be defined as in Theorem \13l and y' be the extension of y. Let 7i = 
{A such that \A\S\ <t-k}, andTi = {B such that \B\S\ <t-k}. Then for all X C 5, MrAz^)h 
and, for all i, Mr^{gi *z^)'^Q . 

Proof. We will first prove that Mji{y')>z and, for all £, M-j-2(g£ * y')— 0- Order the columns and 
rows of Mji{y') by subsets of non-decreasing size. By definition of 7i, any I E 7i of size at least t 
must have |/nS'| > k, and so y'j = 0. Thus 

where M is a principal submatrix of M-pvy)(y).Thus M^ 0, and so M-fi{y')>z 0. 

Similarly, any J € Ti of size at least t — 1 must have \J U {i}n5| > k for every i as well as 
I JriS"! > k, and so, by definition of g£ * y' we must have [g^ * y')j = 0. Thus 

MTM*y') = (^ 

where iV is a principal submatrix of M'p^_^(y^[g£ * y). Thus N>: 0, and so M-j-^{g£ * y')h 0. 

Observe that 7i is closed under shifting by S. By definition of z^ and Lemma \T2\ we thus get 
Mr,{z^)hO. 

Similarly, observe that 72 is also closed under shifting by S. By Lemma[2l we have ge*{P^ *y') = 
P^ *(<?£* y')i ^'^'i so by Lemma [12] again we get Mj:^{g£ * z-^)^^. ■ 

Lemma 15 Let t,y,S,k be defined as in Theorem \ 131 '^^^ u' ^^ ^^^ extension of y. Then for any 
X <ZS: 

1. z^ > 0. 

2. If z^ = then zf = for all \I\ < 2t - 2k. 

Proof. Let 7i be defined as in Lemma [T4l By Lemma [T4l Mt^ (z )>- and z^ is a diagonal element 
of this matrix, hence z^ > 0. 

For the second part, start by considering J C y of size at most t — k. Then J € 7i, and so the 
matrix Mm^j\{z ) is a principal submatrix of Mji^z ), hence is also positive semidefinite. Since 

4 = 0' 

hence Zj = 0. 

Now consider any I QV such that |/| < 2t — 2k, and write / = /1U/2 where |/i| < t — k and 
1^2! <t — k. Mii^j^-\{z-^) is a principal submatrix of Mj--^{z^), hence is also positive semidefinite. 



Since zf = zf = ^,S 



nice 



MiuM{z^)={jx f)^0. 



hence z-f = 0. 

We now have what we need to prove Theorem 1131 

Proof of Theorem 1131 By definition. Lemma [8] and the second part of Lemma [T5l we have 

y\v2t-2k{V) = y'\p2t^2k(V) = Yl ^^\'P2t-2k(V) = Yl 4^^\'P2t-2k{V)- 

xcs xcs 



By Lemma [8] and by definition of y, we have X^xcs-^0^ = 2/0 = 1) &iid tlie terms are non-negative 
by the first part of Lemma \T5\ so y\v2t-2k{V) ^^ ^ convex combination of w^'s, as desired. 

Consider X C 5* such that z^ / 0. By Lemma [H Mr-L{z^)h and Mrzigi * z^)h for ah 
£, and so this also holds for their principal submatrices M-p^_^(^y^{z^) and -/W'-Pt_;,_i(y)(9^ * z^)- 
Scaling by the positive quantity z^ , by definition of w^ this also holds for M-p^_^(-y)(it;^) and 
Mp,_,^,(V)igi *w^)- In other words, w^ \p^^_^^^v) S La*"'' (K). 

Since M'p^_^(^y^{w^^)>z 0, by taking a principal submatrix, we infer M-p^_^^y\s-j{w^')>: 0. Simi- 
larly, M-p^_^(^Y-^{g£*w^^)>z and so M-p^_^(^y\g-j{g£*w^')>z 0. Let g'^ be the constraint of Kj obtained 
from (7£ by setting Xj = w,\ for all j € 5. We claim that for any / C V\S, {gf*z^')j = {g£*z^')j; 

scaling implies that M-p^_^(^y\s){ge * w^') = ^Vt~k{V\S){ge. * w'^^) and we are done. 

To prove the claim, let g(,{x) = Ylj&v ^i^i + ^- Then, by Corollary [lOl g[ = 'Yujevxs '^j^j + (^ + 
EjGX, «i)- Let ^ ^ ^V-S*- We see that 

{gi * w^^)i - {g[ * w^^)i = Y^ aj^fuij} + 2Z "i^fu{J} " 2^ "i^?"- 
By Lemma O w.J.r -^ = w^^ for j € Xj and u;.|Jr 1=0 for j € 5'\Xj. The claim follows. ■ 

5 Upper bound for the Lasserre hierarchy for Knapsack 

In this section, we use Theorem [13] to prove that for the Knapsack problem the gap of La {K) 
approaches 1 quickly as t grows, where K is the LP relaxation of ([T]). First, we show that there is 
a set 5 such that every feasible solution in La {K) satisfies the condition of the Theorem. 

Given an instance (C, V) of Knapsack, Let OPT(C, V) denote the value of the optimal integral 
solution. 

Lemma 16 Consider an instance {C,V) of Knapsack and its linear programming relaxation K 
given by (CP. Let t > I and y G La* (K). Let k < t and 5 = {i G V\vi > OPT{C, V)/k}. Then: 

Y^ Ci>C =^ yi = 0. 

ieins 

Proof. There are three cases depending on the size of /: 

1. |/| <t — 1. Recall the capacity constraint g{x) = C — X^jgy ''«^« — '-'• ^'^ the one hand, since 
^'Pt-i(V)i9 * y)— 0' the diagonal entry (^f * y)j must be non-negative. On the other hand, 
writing out the definition of (^f * y)j and noting that the coefficients q are all non-negative, 
we infer (g * y)f < Cyj — (Xlie/ '^«) V^- -^^t by assumption, X^j^j Q > C. Thus we must have 

yi = o. 

2. t < |/| < 2t - 2. Write / = /1U/2 = I with [/ij, I/2I <t-l and |/inS'[ > k. Then yj^ = 0. 
Since M'p^(^y^>z 0, its 2-by-2 principal submatrix M^j^j^y{y) must also be positive semi-definite. 

and it is easy to check that we must then have yj = 0. 



9 



3. 2t - 1 < |/| < 2t. Write / = /1U/2 = I with |Ii|, I/2I < t and \IinS\ > k. Then yj^ = since 
t < 2t — 2 for aU t > 2. By the same argument as in the previous case, we must then have 



The following theorem shows that the integrality gap of the t^^ level of the Lasserre hierarchy 
for Knapsack reduces quickly when t increases. 

Theorem 17 Consider an instance {C,V) 0/ Knapsack and its LP relaxation K given by (1^. Let 
t > 2. Then 

Value{la^ (K)) < (1 + -^)OPT, 

and so the integrality gap at the t-th level of the Lasserre hierarchy is at most 1 + l/(t — 1). 

Proof. Let S = {i e V\vi > OPT{C,V)/{t - 1)}. Let y £ La* (K). If \InS\ > t - 1, then the 
elements of / n S have total value greater than OPT{C, V), so they must not be able to fit in the 
knapsack: their total capacity exceeds C, and so by Lemma[16]we have y/ = 0. Thus the condition 
of Theorem [13] holds for k = t — 1. 

Therefore, y\'P2{V) is a convex combination oiw^^ withXj C S, thus Value(y) < maxj Value (w'''-*). 
By the first and third properties of the Theorem, we have: 

Yalue{w^') < ^ wj + Value(La^ (Kij). 

By the nesting property of the Lasserre hierarchy. Lemma [H and definition of S, 

Value(Lai {Ki)) < Yalue{K,) < OPT{C - Cost{Xi),V \ S)) + OPT{C, V)/{t - 1). 

By the second property of the Theorem, tD * is in La ^ (K) C K, so it must satisfy the capacity 
constraint, so J2i£X- '^* — "lliei^i ^ C*, so Xi is feasible. Thus: 

Value(y) < max i J" Vj + OPT{C - Cost(X), 1/ \ 5)) 1 + OPT{C, V)/{t - 1) 

feasible XCS \ — ' / 

- \jex ) 

The first expression in the right hand side is equal to OPT{C, V), hence the Theorem. ■ 

6 Conclusion 

We have shown that for Knapsack, an integrality gap of 2 — e persists up to a linear number of rounds 
in the Sherali-Adams hierarchy. This broadens the class of problems for which Sherali-Adams is 
not strong enough to capture the instrinsic difficulity of problems. 

On the other hand, our positive result for Lasserre opens the posibility that lower bounds in the 
Lasserre hierarchy good indicators of the intrinsic dificulty of the problem, thus encourages more 
investigation on the effect of the hierarchy on "easy" problems (SpanningTree, BinPacking, etc.) 

One obstacle along this line is the fact that the second positive semidefinite constraint of the 
hierarchy {M'pu\y{gg * y)>z 0) is notoriously hard to deal with, especially when gi contains many 
variables (in the lowerbounds for /c-CSPs |391l43j. the authors are able to get around this by 
constructing vectors for only valid assignments, an approach that is possible only when all the 
constraints are "small".) Clearly, both lower bounds and upper bounds for the Lasserre hierarchy 
for problems with large constraints remain interesting to pursue. 
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Appendix 

A Full proof of Theorem [5] 

Proof. Let t > 2. Consider the instance K of Knapsack with n objects where Ci = Vi = 1 for 
all i G V and capacity C = 2(1 — e). Let a = C /{n + {t — 1)(1 — e)) and consider the vector 
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y G [0,1]^'^^^ defined by 



yi 



-- 1 

a 
: if 1/1 > 1 



We claim tliat y € SA* {K). Consider any subset U CV sucli tliat \U\ < t. We liave 



M- 



P(U) 



(y) 



Mv,{u){y) 0' 





with M'p^^u){y) 



[l a a 

a a 

a a 

Vq 



a\ 





aj 



Since \U\ < t < n, \U\a < 1, and it is easy to see that this implies M'p^(ij-^{y)'^ 0, and so 
Mp([;)(y)^0. 

Next, let g{x) = C — J2ieV '^«^« ^^^ consider any subset W ^V such that \W\ < t — 1. Again, 
we have 



M„w>«j *y) = {""'''' f"^ I 



M- 



Vi{W) 



{g*y) 



(C-na (C-l)a (C - l)a 
{C-\)a \c-\)a 

\c-\)a {C-\)a 



{C-\)a\ 





\(C -\)a 











(C -\)aj 



Since [VF| < t — 1, by definition of a. we have |VK|(C — l)a < C — na, and it is easy to see that this 
implies M-p^ty^-^^g * y)^ 0, and so M-p^y^-^i^g * y)^ 0. Thus y G SA* {K\ 

The integer optimum has value 1, so the integrality gap is at least the value of y, which is 
na = 2(1 — e)/(l + {t — 1)(1 — 2e)/n). The supremum over all e is 2/(1 + (t — l)/n), and the 
supremum of that over all n is 2, so the integrality gap is at least 2. 

On the other hand, it is well-known that the base linear program K has value at most 20PT 
(that is an immediate consequence of Lemma [TJ, hence, by the nesting property, every linear 
program in the hierarchy has integrality gap exactly equal to 2. ■ 

B A linear formulation of the Sherali- Adams hierarchy 

For any constraint ge{x) > in the definition of the base polytope K and any subsets I,J '^ V, 
the following constraint is a consequence of the fact that x £ [0, 1]": 



d^)llxillil-xj)>0. 
iei jeJ 



(6) 



If X is indeed integral, then xf = Xi for any k > 1. Thus, the constraint obtained by expanding ([6]) 
and replacing x^ by Xi holds in P and can be added to strengthen the relaxation. However, this 
constraint is not linear. To preserve the linearity of the system, each product Yiiei ^« ^^ replaced 
by a variable yj. 

In addition, to keep the number of variables from growing exponentially, we restrict ourselves 
to only variables y/ such that |/| < t. By this, we "lift" the polytope if to a polytope SA* (K) C 
[0,1]^'W. 
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Definition 18 Let K he a polytope defined as in equation\^ For any 1 <t <n, the t-th Sherali- 
Adains lifted polytope SA {K) is defined by 

SA*(K) = {y e7't(N)|y0 = 1, and g[j^j{y) > for any i and I, J Q V s.t. \IUJ\ < t - l} 

where g'u j{y) is obtained by: 

1. multiplying gi{x) by Hie/ ^i l^jejC^ ~ ^j)' 

2. expanding the result and replacing each x^ by Xj/ and 

3. replacing each HieS ^« ^2/ ^5- 

We say that a point x G [0, 1]^ belongs to the t-th Sherali-Adams polytope sa* (K) iff there 
exists a y ^ SA* (K) such that y^ij = Xi for all i ^ V. 

In particular, in the case of Knapsack, SA* (K) is the set of all points in [0, 1]^*(^) that satisfy 
the following constraints for any I,JQV such that InJ = and |/| + | J| < t — 1: 

n 

J]c,^(-l)l^ly,uLu« <C5](-l)l^ly,uL, (7) 

i=l LCJ LCJ 

and 

< j;(-i)'^'y7uLu{i} < j;(-i)"''y/uL, V^ E V. 

LCJ LCJ 

For a proof that this definition is equivalent to Definition O we refer the reader to Laurent's 
paper [32] , 
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